Helping deaf and hard-of-hearing people by combining augmented reality and speech technologies

Recently, many studies have shown that the Augmented Reality (AR), Automatic Speech Recognition (ASR) and Text-to-Speech Synthesis (TTS) can be used to help people with disabilities. In this paper, we combine these technologies to make a new system, called "ASRAR", for helping deaf people. This system can take a narrator's speech and convert it into a readable text, and show the text directly on AR displays. Since most deaf people are unable to make meaningful sounds, we use a TTS system to make the system more usable for them. The results of testing the system show that its accuracy is over 85 percent, using different ASR engines, in different places. The results of testing TTS engines show that the processing time is less than three seconds and the spelling of correct words is 90 percent. Moreover, the result of a survey shows that more than 80 percent of deaf people are very interested in using the ASRAR system for communication.